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I. Real party in Interest 

The real party in interest is International Business Machines Corporation, the 
assignee. 



II. Related Appeals and Interferences 

There are no other appeals or interferences known to appellant, appellant's 
representative or assignee that will directly affect or be directly affected by or have a 
bearing on the Board's decision in the pending appeal. 



III. Status of Claims 

Claims 2, 3, 5, and 6 are currently pending. They have been finally rejected. 
Claims 1, 4, and 7 have been cancelled. 

The pending rejected claims that form the basis of this appeal are reproduced in the 
attached Appendix. 



IV. Status of Amendments 

An Amendment After Final was filed March 23, 2004 canceling claims 1, 4, and 7 
and presenting arguments to overcome the Examiner's rejection of the remaining pending 
claims, claims 2, 3, 5, and 6. A Supplemental Amendment After Final To Correct 
Previously Filed Non-Compliant Amendment After Final was filed April 15, 2004 to 
correct the status of claims 2, 3, and 5 to be "previously amended" and not "original". 
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V. Summary of the Invention 

The present invention encompasses a method and computer system for interacting 
between participants in a network of computers. More specifically, a video camera is used 
to capture an actual physical gesture, such as a wave, a shoulder shrug, a head nod, etc., 
made by a participant. Image processing software analyzes the captured video images of 
the actual physical gesture, received as input, of a participant (page 13, lines 3-5, 9-20; 
page 19, line 14 to page 20, line 6). When a gesture is depicted, the computer system uses 
a corresponding graphic or text translation, such as an emoticon or a text description or 
animation of an avatar, and inserts the translation into the participant's dialogue in the live 
chat session in accordance with the command interface to the chat room software. In this 
way, a representation of the gesture is automatically generated and can be inserted within a 
communication from one participant to each of the other participants in an on-line chat 
session within the network (page 1, lines 10-16; and page 7, line 10 to page 8, line 4). 

In one embodiment, after analyzing the captured video images of actual physical 
gestures, each of the physical gestures are associated with separate commands of an 
application program interface for communicating in real time between the participants. 
An associated command associated with the actual physical gesture from the analyzed 
images is transmitted to the application program interface to send a representation of the 
actual physical gesture within the real time communication for communicating between 
the participants. That is, a chat room command is determined, e.g., a graphic or text 
representation of the gesture, and sent via the communication from the participant using 
the chat software (Fig. 3; page 12, lines 24-27; page 13, line 21 to page 15, line 15). 

In another embodiment, after analyzing successive video images that are received 

as input from a camera capturing video of an actual physical gesture made by a participant 

(page 13, lines 3-5, 9-20; page 19, line 14 to page 20, line 6), a state of the actual physical 

gesture made by the one participant is determined (page 13, line 27, page 14, line 5 to 

page 15, line 10; page 15, lines 17-26; page 17, lines 19-27; page 20, lines 7-11). A table 

is accessed for an action associated with the determined state of the actual physical gesture 
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(Fig. 3; page 17, line 28 to page 18, line 8; page 20, lines 11-15). A command for the 
action is automatically generated to a software program enabling a real time 
communication between the participants thereby sending a representation of the actual 
physical gesture within the real time communication (page 20, lines 1 1-27). 

VI. Issues Presented for Review 

A. Whether the Examiner's rejection of claim 3 as being anticipated by Kellner 
(6,539,099) under 35 USC 102(e) is well-formed. 

B. Whether the Examiner's rejection of claims 2, 5, and 6 as being unpatentable over 
Kellner (6,539,099) in view of Hatlelid (6,404,438) under 35 USC 103(a) is well-formed. 

VII. Grouping of Claims 

The rejected claims do not stand or fall together. 
Claim 3 stands or falls by itself as a Group I claim. 

Claims 2, 5, and 6 all stand or fall together as Group II claims. However, claims 2, 
5, and 6 will stand together with claim 3 if claim 3 stands. In addition, claims 2, 5, and 6 
can stand even if claim 3 should fall. 

VIII. Argument 

A. Whether the Examiner's rejection of claim 3 as being anticipated by Kellner 
(6,539,099) under 35 USC 102(e) is well-formed. 
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Kellner discloses a character image that is read into memory that represents a 
character that a user wishes to be for the duration of a visual chat. Kellner further 
discloses using a video camera to receive video images of image data of a person. An 
image of a person is tracked, and portions of the image are extracted from the video 
images. Preferably, the extracted portions are features of the person. The extracted 
portions, or features, are blended into corresponding areas of the character image, such 
that the features of the blended character image match the features of the person, and 
change as the features of the person change. (See abstract; Figs. 3-7; column 6, lines 8-27; 
column 8, lines 8-60.) 

More specifically, Kellner discloses the processing of the sampled feature 
locations at column 5, line 66 to column 6, line 26, including the statement that "the 
system transmits processed information concerning the sampled feature locations to the 
decoder systems of other visual chat users. Thus, rather than transmitting the entire video 
image to other users, only data concerning the sampled feature locations is transmitted to 
other users." The processing of the sampled features is further described in detail starting 
with column 6,line 28. This description involves "person pixels" and other "data values". 

In the advisory Action, the examiner states: 

"Each gesture received, whether the same, similar or completely different 
from another gesture, will need to be processed using different, i.e., separate, 
commands of an API in order to render the image to represent the actual physical 
gesture that occurred. For example, two different gestures, i.e., waving a hand 
and smiling, will require different commands/instruction to be used by the system 
in order to alter the image to represent the actual physical gesture. Thus, the 
commands and instructions necessary to show a hand waving and a face smiling 
will be different and separate. Therefore, separate gestures, i.e., hand waving or 
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face smiling, requires separate commands/instructions, i.e. instructions to alter 
the image for a hand wave and a smile." 

Applicants assert that the Examiner's position is not well-formed. The 

Examiner is describing Applicants' invention and not Kellner. The Examiner points to no 

place in Kellner that supports the statements made by the Examiner. Kellner alters the 

image of features as a result of the data collected, processed, and sent. Kellner analyzes 

the image at the pixel level and processes the pixel data. To the contrary, Applicants 1 

claimed invention associates a total gesture with separate commands. 

Thus, Kellner does not teach nor suggests "associating each of a plurality of 

gestures to separate commands of an application program interface for communicating in 

real time between the participants; and transmitting an associated command associated 

with the actual physical gesture to the application program interface for communicating 

between the participants." At column 8, lines 26-49, Killner merely describes that the 

features are to be blended into the character image. Killner does not teach or suggest that 

each gesture is associated with a separate command, as claimed in Applicants' claimed 

invention. With reference to column 5, line 65 to column 6 line 24, Kellner teaches that 

data ("processed information" or "data concerning the sampled feature locations") not 

"commands" are being transmitted to the decoder systems of other chat users. Since 

Kellner does not associate a gesture with a separate command, these commands are not 

being transmitted. Kellner merely processes data from the video image; a given physical 

gesture is not being associated with a separate command. 
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A claim is anticipated only if each and every element as set forth in the claim is 
found, either expressly or inherently described in a single prior art reference. Verdegaal 
Bros. V. Union Oil Co. Of California , 814 F.2d 628, 631, 2 USPQ2d 1051, 1053 (Fed. 
Cir. 1987). 

Since Kellner does not teach each and every claim limitation of Applicants' 
claimed invention, Kellner does not anticipate Applicants' claimed invention under 35 
USC 102(e). 

B. Whether the Examiner's rejection of claims 2, 5 9 and 6 as being unpatentable 
over Kellner (6,539,099) in view of Hatlelid (6,404,438) under 35 USC 103(a) is well- 
formed. 

To establish prima facie obviousness of a claimed invention, all of the claim 
limitations must be taught or suggested by the prior art. MPEP 2143.03. See Also, In re 
Rovka. 490 F.2d 580 (CCPA 1974). 

For the same reasons as discussed above with respect to ISSUE A and the Group I 
claim, claims 2, 5 and 6 of the Group II claims are patentable since neither Killner nor 
Hatlelid teach or suggest the claimed element of "automatically generating a command for 
the action to a software program enabling a real time communication between the 
participants thereby sending a representation of the actual physical gesture within the real 
time communication. 1 ' Neither Hatlelid nor Killner automatically generate a command 
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for the action (where the action is associated with a determined state of an actual 
physical gesture). Killner sends data, not a command, as discussed above; and Hatlelid 
has behavioral movements for gesture ids, but not commands for an actual physical 
gesture. 

Furthermore, neither reference accesses a table for an action associated with the 
determined state of the actual physical gesture. The examiner admits (page 5 of the Final 
Office Action) that Kellner fails to explicitly teach determining a state of the actual 
physical gesture and accessing a table for an action associated with the determined state. 

The examiner states (Final Office Action page 5) that Hatlelid teaches a system for 
determining if a trigger is a first occurrence or not, upon which a different behavioral 
movement for a given trigger exists in a table depending on the occurrence number, i.e., 
first occurrence or not, and cites Fig. 12a and col. 28, lines 18-26. 

This statement of the Examiner is not well-formed. Fig 12a merely shows a list 
box of phrases linked to behavioral movements. It does not show "an action associated 
with a given state of the actual physical gesture" as claimed in Applicants 1 claimed 
invention. In Hatlelid, at column 27, lines 27-30, an assigned weight to a new rule merely 
indicates the frequency in which a given behavioral movement is to be animated when a 
gesticulatory trigger is encountered in an utterance. Thus, Hatlelid again does not teach 
nor suggests "an action associated with a given state of the actual physical gesture" as 
claimed in Applicants* claimed invention. In column 28, a threshold value is discussed 
which refers to the number of rules, not the "state of an actual physical gesture." As stated 
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at column 28, lines 53-56 "a new general rule 1 1 16 is not created unless the number of 

existing rules 1116 that could serve as a basis for the new generated rule 116 exceeds a 

threshold." What is being counted in Hatlelid is the number of existing rules, not an 

"actual physical gesture" as claimed in Applicants 1 claimed invention. 

Since neither Kellner nor Hatlelid, alone or in combination, teach or suggest each 
and every claim limitation of Applicants' claimed invention, the combination of references 
does not make Applicants' claimed invention obvious under 35 USC 103(a). 



It is therefore respectfully requested that the Examiner's rejection of Claim 3 
under 35 USC Section 102(e), and the rejection of Claims 2, 5, and 6 under 35 USC 
Section 103 be reversed. It is respectfully submitted that the claims remaining in the 
Application are patentable and allowance of these claims to Appellants is respectfully 
requested. 

Respectfully submitted, 



Marilyn Smith Dawkins 
Attorney for Applicants 
Registration No. 3 1 , 1 40 
(512) 823-0094 
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APPENDIX 



1. (CANCELLED) 

2. (PREVIOUSLY AMENDED) A method for interacting between participants 
through a network of computers, comprising: 

analyzing successive video images received as input from a camera capturing 
video of at least one of the participants for an actual physical gesture made by the one 
participant; 

determining a state of the actual physical gesture made by the one participant; 

accessing a table for an action associated with the determined state of the actual 
physical gesture; and 

automatically generating a command for the action to a software program enabling 
a real time communication between the participants thereby sending a representation of 
the actual physical gesture within the real time communication. 

3. (PREVIOUSLY AMENDED) A method for communicating between 
participants through a network of computers, comprising: 

analyzing images of at least one of the participants received as input for an actual 
physical gesture; 

associating each of a plurality of physical gestures to separate commands of an 
application program interface for communicating in real time between the participants; 
and 

transmitting an associated command associated with the actual physical gesture 
from the analyzed images to the application program interface to send a representation of 
the actual physical gesture within the real time communication for communicating 
between the participants. 
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4. (CANCELLED) A computer system having means for interacting between 
participants through a network of computers, comprising: 

means for analyzing successive video images received as input from a camera 
capturing video of at least one of the participants for an actual physical gesture made by 
the one participant; and 

means for automatically generating a command to a software program enabling a 
real time communication between the participants to send a representation of the actual 
physical gesture within the real time communication. 

5. (PREVIOUSLY AMENDED) A computer system having means for interacting 
between participants through a network of computers, comprising: 

means for analyzing successive video images received as input from a camera 
capturing video of at least one of the participants for an actual physical gesture made by 
the one participant; 

means for determining a state of the actual physical gesture made by the one 
participant; 

means for accessing an associative mapping for an action associated with 
the determined state of the actual physical gesture; and 

means for automatically generating a command for the action to a software 
program enabling a real time communication between the participants thereby sending a 
representation of the actual physical gesture within the real time communication. 

6. (ORIGINAL) The system of claim 5 wherein the associative mapping is a 

table. 

7. (CANCELLED) 
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